The Wave of Risk Perception

I had an interesting chat with a former member of my team recently. We were in the pub and she was sharing with me her woes in a new role. The problem, as she explained it, was that the members of the leadership team in her company were pushing back against her efforts to devote development time to improving existing features, robustness and stability of the product. They were solely interested in new features. The issue seemed to be that they didn't have an appreciation of the levels of risk involved in their existing product. To help her to understand the reasons for their position I drew up a diagram that I've used in a few talks to explain the perception of risk. The conversation reminded me that I'd not written about this, so here goes...

Destined to be Different

It is not a coincidence that in many situations people close to software development, particularly testers, have a very different perception of risk than some leadership roles. One of my favourite historical blog posts is this one on why testers and business leaders inevitably differ in their perception of risk. I recommend reading that one for a deeper exploration, but to summarise here:-

Humans are susceptible to the 'Availability Heuristic' that leads us to attribute higher risk to outcomes which we can recall experiences or stories around. Therefore roles with greater visibility of issues that have occurred through a process, or of software problems in general, will attribute higher risk to a software release than those who have not
Research such as this paper has additionally shown that managers who don't have a detailed understanding of negative outcomes will aggregate multiple separate negative possibilities into one 'risk of failures' which they then attribute a lower level of risk to than is appropriate due to lack of availability.
The 'Affect Heuristic' means that we naturally attribute lower risk to things that we associate a strong positive outcome to, and vice-versa, so a manager focusing emotionally on the benefits of a software release for customers will typically attribute lower risk to the release than those roles with a focus on the challenges of technical delivery

These three factors combined give good cause to expect leadership roles to have a much lower perception of risk around a software delivery than testers and other development roles.

The Wave of Perceived risk

Understanding differences in risk perception is all well and good, however unless it is in understanding and recognising the potential outcomes of this disparity that the theory becomes practically useful. What would we expect to see as a result of this somewhat inevitable difference in perspectives?

When creating a talk around risk and the reasons that I came up with the 'risk questionnaire' I used the following diagram as a means to represent the scenario that can arise when there is a strong disparity of risk perception between testers and management. The idea here was to represent perceived risk over time in a tester and a product role on a new development and to explain the shape of testing that we would expect to see as a result.

Initially when commencing development what we expect to see is a large difference in their perceived levels of risk around the same situation. The perception of risk in management is low and decisions on approach and expected pace of development are driven based on this. The tester is aware of the lack of rigour in place and will have an elevated perception of risk driven by their experience of the many issues that could surface.

Inevitably one of these potential problems goes bang and the manager is suddenly and painfully aware of a specific risk.

Additional effort is justified to tackle 'the problem'. This is likely to be in the form of a process improvement or the creation of some new tests, however the scope of this work will in all likelihood be limited to covering the issue that emerged, possibly in excess of an appropriate level given the testers elevated perception of risk.

The manager is satisfied and returns to a having a lower perception of the risk situation, however knowledge of the problem does give some increased awareness of the chances of hitting problems. The Tester, on the other hand, knows that only one of the many areas of potential concern has been mitigated and their confidence is only slightly increased due to their maintained perception of the other potential negative outcomes in their process.

Inevitably another issue cross up in another area with similar consequences

over time the encountering of issues and improving testing reaches a point where, whilst not necessarily aligned, the perception of risk between tester and manager is closer and in balance with a consensus around the level of testing needed and the resulting chance of hitting issues.

The testing is likely to be formed around pockets of high (potentially excessive) coverage where risks have manifested themselves, with lighter coverage in other areas.

Surfing the Wave

I was slightly nervous when I first presented this at a conference, however I've presented this a few times now and had many testers confirming that this looks very much like the way that testing evolved in their organisation. At the last talk one audience member said it was the best way of representing this situation that he had seen.

The same person asked an interesting question - given the choice would you reduce the amplitude or the frequency of the curve? To rephrase - is the way to tackle the problem to reduce the number of issues or the difference in perception to start with? For me the two are separate but go hand in hand, and it is necessary to address the second to justify focussing on the first. Working towards a more common perception of risk between roles will not in itself reduce the problems encountered. Without addressing this disparity, however, it can be hard for testers to establish and justify a level of rigour around development required to do so. Such a change for a manager with a low perception of risk would appear to be an increase in development time and cost for little benefit. If we want backing to improve testing, without just bouncing from issue to issue to justify point improvements, then helping to establish a more consistent perception of risk is clearly a key area to focus.

The good news here is that, from our understanding of the causes of the difference in perception, we have a great starting point in understanding this might be achieved. I identified at the start of this post three biases that impact our perception of risk that we can act to influence.

Tell stories to convey risk information. Human risk perception simply isn't driven by the logical brain processes that deal with facts and figures. Risk perception is influenced far more through 'availability' of stories and experiences that convey the potential for negative outcomes. My personal experience supports this - one excellent example involved a conversation with a CEO once around improving data security that I was expecting to be a challenge, however he had just recently read an article on a data breach from a similar company which made headline news, and he was therefore very open to addressing the risks in that area.
Don't aggregate all conversations on issues into 'bugs'. Discussing different potential problem areas separately will help to establish a more realistic understanding of the multiple negative outcomes that could arise in our development. Actually discussing these collectively as bugs could result in an under-appreciation of the different risks in those not close to the details of the development.
Include testers early in value conversations - Testers are often excluded from early conversations around developments, and are only brought in when the solutions are scoped. Yet it is in those early conversations that the real value of the development is established and discussed. It carries then that one positive way to avoid the curve by influencing their own risk perception is by getting themselves involved from the start and understanding the value that each development could deliver.

It is worth remembering here that our focus needs to be on establishing a realistic and consistent perception of risk across roles, which involves more than just increasing awareness of risk in management. Despite the superpowers to find issues and save projects, testers are only human and just as susceptible to biases in their risk perception as everyone else and the tester's default position is as likely to involve over-estimation of risk as management roles are to under-estimate.

A key element to the testing role is to understand the biases that can influence the perception of risks in our own roles as well as others. Understanding these can provide the key to being able to have genuinely influential conversations with business leaders in ways that will effectively convey risks in a way that is meaningful to them, and hopefully avoid the painful process of progressing up the wave that comes when different roles start from very different perspectives.

https://www.pexels.com/photo/action-beach-fun-leisure-416676/